System for Support Vector Machine Prediction

ABSTRACT

A computer-implemented method is disclosed for image recognition and other applications. The method employs an SVM model and can reduce false negatives and increase recognition accuracies by raising the sample-to-support-vector ratio.

TECHNICAL FIELD

This application relates to improved image recognition methods forautomated imaging and other applications.

BACKGROUND OF THE INVENTION

The wide adoption of smart phones and digital cameras has caused anexplosion in the number of digital images. Digital images can be viewed,shared over the Internet, and posted on mobile applications and socialnetworks using computer devices. Images can also be incorporated intophoto products such as photographic prints, greeting cards, photo books,photo calendars, photo mug, photo T-shirt, and so on. In electronicforms or on physical products, images are often mixed with text andother design elements, and laid out in a particular fashion to tell astory.

Digital images often contain significant objects and people's faces.Creating photo blogs or high-quality image products, such as photobooksand greeting cards, naturally requires proper consideration of thoseobjects and people's faces. For example, important people such as familymembers should have their faces prominently shown in image productswhile strangers' faces should be minimized. In another example, whilepictures of different faces at a same scene can be included in animage-based product, the pictures of a same person at a same sceneshould normally be trimmed to allow only the best one(s) to bepresented. Examples of significant objects include people's clothing,daily objects such as furniture, a birthday cake, and a soccer ball, andnatural or man-made landmarks.

Faces need to be detected and group based on persons' identities beforethey can be properly selected and placed in image products. Mostconventional face detection techniques concentrate on face recognition,assuming that a region of an image containing a single face has alreadybeen detected and extracted and will be provided as an input. Commonface detection methods include knowledge-based methods;feature-invariant approaches, such as the identification of facialfeatures, texture and skin color; template matching methods, both fixedand deformable; and appearance based methods. After faces are detected,face images of each individual can be categorized into a groupregardless whether the identity of the individual is known or not. Forexample, if two individuals Person A and Person B are detected in tenimages. Each of the images can be categorized or tagged one of the fourtypes: A only; B only, A and B; or neither A nor B. Algorithmically, thetagging of face images require training based one face images of knownpersons (or face models), for example, the face images of family membersor friends of a user who uploaded the images.

To save users' time, technologies have been developed to automate thecreation of photo-product designs. These automatic methods are facingincreased challenges as people take more digital photos. A singleaverage vacation trip nowadays can easily produce thousands of photos.Automatic sorting, analyzing, grouping, and laying out such a greatnumber of photos in the correct and meaningful manner are an immensetask.

There is therefore a need for more accurately recognizing and groupingface images and other objects and incorporating them into photo-productdesigns and other imaging applications.

SUMMARY OF THE INVENTION

Various machine learning methods have been used to recognize andcategorize faces or objects in images. One commonly used method issupport vector machine (SVM). A common problem in SVM and other machinelearning methods is that their recognition accuracies are low when thesample set is small. The cause for this problem is that there aretypically a large number of support vectors; a small sample set leads todata over-fitting. Such sample bias results in recognition errors in theform of false positives and false negatives.

The present application discloses novel methods that increase the ratioof sample number to support vector number in an SVM model, which canreduce false negatives and increase recognition accuracies.

In a general aspect, the present invention relates to acomputer-implemented method that includes selecting a kernel and kernelparameters for a first Support Vector Machine (SVM) model comprising SVMsupport vector number of support vectors, and testing the first SVMmodel on a feature matrix T to produce false positive (FP) data set andfalse negative (FN) data set by a computer system, wherein the featurematrix T includes n feature vectors of length m, wherein n and m areinteger numbers. The method includes copying the feature matrix T toproduce a feature matrix T_best comprising T_best sample number ofsample points, checking if a ratio (T_best sample number)/(SVM supportvector number) is above a threshold for the first SVM model on T_best,and if the ratio is above the threshold, performing SVM predictionsusing the first SVM model on the feature matrix T_best.

Implementations of the system may include one or more of the following.The n feature vectors can be based on faces or objects in sample images,wherein the first SVM model can be used on the feature matrix T_best toclassify the faces or the objects in sample images. Thecomputer-implemented method can further include creating animage-product design based on the faces or the objects in the imagesclassified by the first SVM model using the feature matrix T_best. Ifthe ratio (T_best sample number)/(SVM support vector number) is notabove the threshold for the first SVM model on T_best, repeating thesteps of selecting, testing, automatically removing, retraining, andchecking to find a second SVM model having (T_best sample number)/(SVMsupport vector number) ratio that exceeds the threshold. Thecomputer-implemented method can further include performing SVMpredictions using the second SVM model on the feature matrix T_best. Thecomputer-implemented method can further include: if a SVM model having(T_best sample number)/(SVM support vector number) ratio that exceedsthe threshold is not found, selecting a third SVM model that yields ahighest (T_best sample number)/(SVM support vector number) ratio in thestep of repeating the steps of selecting, testing, automaticallyremoving, retraining, and checking; and performing SVM predictions usingthe third SVM model on the feature matrix T_best. The threshold for the(T_best sample number)/(SVM support vector number) ratio can be ten orhigher. The computer-implemented method can further include applyingGrid Search on the feature matrix T to find optimal kernel parametersbefore the step of testing the first SVM model on a feature matrix T.The feature vectors can be based on faces or objects in images; thecomputer-implemented method can further include classifying the objectsand the faces in the images by performing SVM predictions on the featurematrix T_best. The computer-implemented method can further includeautomatically creating designs for photo products based on the objectsand faces classified by performing SVM predictions on the feature matrixT_best.

The computer-implemented method can further include selecting a featurevector v from the FP data set to remove from the feature matrix T_bestto produce a feature matrix T_v, retraining the first SVM model on thefeature matrix T_v to produce a fourth SVM model, calculating a benefitfunction to produce a benefit function value, wherein the benefitfunction is dependent of differences between false positives, falsenegatives, and numbers of support vectors generated respectively onT_best by the first SVM model and on T_v by the fourth SVM model,removing the feature vector v from the feature matrix T_best to obtainan updated feature matrix T_best if the benefit function value meets apredetermined criterion; and performing SVM prediction using the fourthSVM model on the updated feature matrix T_best. The benefit function canbe dependent more strongly on the difference between the false negativesthan on the difference between the false positives. The differencebetween false negatives can have a first weighting coefficient in thebenefit function, wherein the difference between false positives canhave a second weighting coefficient in the benefit function, wherein thefirst weighting coefficient can be larger absolute value than the secondweighting coefficient. The n feature vectors can be based on faces orobjects in sample images, wherein the fourth SVM model can be used onthe feature matrix T_best to classify the faces or the objects in sampleimages. The computer-implemented method can further include creating animage-product design based on the faces or the objects in the imagesclassified by the fourth SVM model using the feature matrix T_best. Thecomputer-implemented method can further include repeating the steps ofselecting a feature vector v, training the first SVM model, performingSVM predictions, calculating a benefit function, and removing thefeature vector v from T_best by selecting and removing a differentfeature vector corresponding to the FP data set from the feature matrixT_best, wherein SVM predictions are performed using the feature matrixT_best that gives a highest best function value. All the feature vectorscorresponding to the FP data set can be evaluated by calculating acorresponding benefic function, where are respectively determined to beremoved from the T_best or not depending on a value of the correspondingbenefic function. The feature vector v is not removed from the featurematrix T_best if the value of the benefit function does not meet thepredetermined criterion. The feature vectors can be based on faces orobjects in images, the computer-implemented method can further includeclassifying the objects and the faces in the images by performing SVMpredictions on the feature matrix T_best.

These and other aspects, their implementations and other features aredescribed in detail in the drawings, the description and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a network-based system for producingpersonalized image products and photo blogs and shares compatible withthe present invention.

FIG. 2 is a flow diagram illustrating a high-recall SVM method withincreased recognition accuracy in accordance with the present invention.

FIG. 3 is a flow diagram illustrating a subtractive high-recall SVMmethod for reducing false negatives and increasing recognition accuracyin accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, a network-based imaging service system 10 operatedby an image service provider such as Shutterfly, Inc. enables users 70,71 to store, organize, and share images via a wired network or awireless network 51. The network-based imaging service system 10 alsoallows users 70, 71 to design image products.

The network-based imaging service system 10 includes a data center 30,one or more product fulfillment centers 40, 41, and a computer network80 that facilitates the communications between the data center 30 andthe product fulfillment centers 40, 41. The data center 30 includes oneor more servers 32 for communicating with the users 70, 71, a datastorage 34 for storing user data, image and design data, and productinformation, and computer processor(s) 36 for rendering images andproduct designs, organizing images, and processing orders. The user datacan include account information, discount information, and orderinformation associated with the user. A website can be powered by theservers 32 and can be accessed by the user 70 using a computer device 60via the Internet 50, or by the user 71 using a wireless device 61 viathe wireless network 51. The servers 32 can also support a mobileapplication to be downloaded onto wireless devices 61.

The network-based imaging service system 10 can provide products thatrequire user participations in designs and personalization. Examples ofthese products include the personalized image products that incorporatephotos provided by the users, the image service provider, or othersources. In the present disclosure, the term “personalized” refers toinformation that is specific to the recipient, the user, the giftproduct, and the occasion, which can include personalized content,personalized text messages, personalized images, and personalizeddesigns that can be incorporated in the image products. The content ofpersonalization can be provided by a user or selected by the user from alibrary of content provided by the service provider. The term“personalized information” can also be referred to as “individualizedinformation” or “customized information”.

Personalized image products can include users' photos, personalizedtext, personalized designs, and content licensed from a third party.Examples of personalized image products may include photobooks,personalized greeting cards, photo stationeries, photo or image prints,photo posters, photo banners, photo playing cards, photo T-shirts, photomugs, photo aprons, photo magnets, photo mouse pads, a photo phone case,a case for a tablet computer, photo key-chains, photo collectors, photocoasters, photo banners, or other types of photo gift or novelty item.The term photobook generally refers to as bound multi-page product thatincludes at least one image on a book page. Photobooks can include imagealbums, scrapbooks, bound photo calendars, or photo snap books, etc. Animage product can include a single page or multiple pages. Each page caninclude one or more images, text, and design elements. Some of theimages may be laid out in an image collage.

The user 70 or his/her family may own multiple cameras 62, 63. The user70 transfers images from cameras 62, 63 to the computer device 60. Theuser 70 can edit, organize the images on the computer device 60, whichcan include a personal computer, a laptop, tablet computer, a mobilephone, etc. The cameras 62, 63 include a digital camera, a camera phone,a video camera, as well as image capture devices integrated in orconnected with in a computer device, such as the built-in cameras inlaptop computers or computer monitors. The user 70 can also printpictures using a printer 65 and make image products based on the imagesfrom the cameras 62, 63.

Images can be uploaded from the computer device 60 and the wirelessdevice 61 to the server 32 to allow the user 70 to organize and renderimages at the website, share the images with others, and design or orderimage product using the images from the cameras 62, 63. If users 70, 71are members of a family or associated in a group (e.g. a soccer team),the images from the cameras 62, 63 and the mobile device 61 can begrouped together to be incorporated into an image product such as aphotobook, or used in a blog page for an event such as a soccer game.

Products can be ordered by the users 70, 71 based on the image productdesigns. The physical products can be manufactured by the printing andfinishing facilities 40 and 41 and received by recipients at locations180, 185. The recipients can also receive digital versions of theimage-product designs over the Internet 50 and/or a wireless network 51.For example, the recipient can receive, on her mobile phone, anelectronic version of a greeting card signed by handwritten signaturesfrom her family members.

For the fulfillments of personalized image products, the productfulfillment center 40 can include one or more printers 45 for printingimages, finishing equipment 46 for operations such as cutting, folding,binding the printed image sheets, and shipping stations 48 for verifyingthe orders and shipping the orders to recipients 180 and 185. Examplesof the printers 45 include can be digital photographic printers, offsetdigital printers, digital printing presses, and inkjet printers. Thefinishing equipment 46 can perform operations for finishing a completeimage-based product other than printing, for example, cutting, folding,adding a cover to photo book, punching, stapling, gluing, binding, andenvelope printing and sealing. The shipping stations 48 may performtasks such as packaging, labeling, package weighing, and postagemetering.

The creation of personalized image products can take considerable amountof time and effort. To save users' time, technologies have beendeveloped by Shutterfly, Inc. to automate the creation of photo-productdesigns. Given the thousands of photos that people now often take justfrom one occasion, automatic sorting, analyzing, grouping, curating, andlaying out a large number of photos have become increasinglychallenging. Accurately recognizing faces and objects in photos areessential to the technologies for automated photo-product designs. Tothis end, an improved machine learning method based on SVM is describedbelow, which has helped the recognition accuracies in classifying andrecognizing objects or faces, especially when only a small number ofsample objects or faces are available in those images.

A common problem in SVM and other machine learning methods is thatrecognition accuracy is low when the sample set is small. The cause forthis problem is that there are typically a large number of supportvectors; a small sample set can lead to data overfitting, resultingfalse positives and false negatives. Precision and Recall are twometrics for measuring classifier output quality. Higher Precision can beachieved by reducing false positives, while higher Recall is obtained byreducing false negatives.

In the presently disclosed methods, the over fitting problem is reducedand recognition accuracy improved by tackling one of, rather than, bothhigher Precision and higher Recall goals. Since False Negatives are animportant source for recognition inaccuracies, the disclosed novel SVMmethods have been developed by focusing on achieving higher Recall inclassifying face and other objects.

A SVM model includes a kernel function, kernel parameters used fortransforming predicted data, and a set of support vectors which are usedto separate the transformed data into True and False predictions.

In some embodiments, FIG. 2 shows a first sub-flow of the disclosedmethods, in which the kernel and kernel parameters are optimized in theSVM model to increase the ratio of sample number to support vectornumber to above a threshold in a feature matrix.

Referring to FIGS. 1 and 2, a computer processor selects kernelparameters for Support Vector Machine (SVM) using a feature matrix Tcomprising n feature vectors each having a length m (step 200). Inpattern recognition and machine learning, a feature vector is ann-dimensional vector of numerical features that represent some objects(e.g. a face image or other objects as described above). Representinghuman faces or objects by numerical feature vectors can facilitateprocessing and statistical analysis of the human faces or the objects.The vector space associated with these feature vectors is often calledthe feature space.

A feature matrix comprising all feature vectors represents the entiredata set used for testing, training, and validation of SVM models. Theentire data set is divided into test set, and train set. At every pointon the grid for grid search, a cross validation set is randomly sampledfrom the train set. The SVM model is trained on train set or the crossvalidation set. Precision and Recall are evaluated by comparing SVMprediction on the cross validation set with the Ground Truth labels onthat set. The SVM model with the best precision or best recall is beingchosen from the grid search.

The selection of kernel parameters for SVM is conducted on a test dataset. Kernel methods require only a user-specified kernel, i.e., asimilarity function over pairs of data points in raw representation.Kernel methods employ kernel functions to operate in a high-dimensional,implicit feature space without ever computing the coordinates of thedata in that space, but rather by simply computing the inner productsbetween the images of all pairs of data in the feature space. Thisoperation is often computationally cheaper than the explicit computationof the coordinates. The computer processor can be implemented by thecomputer processor 36, the computer device 60, or the mobile device 61.

In the disclosed high-recall SVM method, the computer processor appliesGrid Search on the feature matrix T to find optimal kernel parameters(step 210). A set of pre-prepared “Ground Truth” labels for all featurevectors. The computer processor compares the Ground-Truth labels to theSVM prediction vector to find TP, TN, FP, FN data sets. The SVM modelbased on the kernel and the optimized kernel parameters is then testedon the entire data set, the feature matrix T, to produce false positive(FP) and false negative (FN) data sets (step 220).

The feature matrix T is then copied to T_best (step 230), and the newSVM model is evaluated on T_best. The computer processor checks if theratio (T_best sample number)/(SVM support vector number) is above athreshold for the new SVM model on T_best by the computer processor(step 240). For example, the threshold for the ratio can be set at 10 orhigher, meaning that it is desirable for the sample number to be atleast ten times of the support vector number in the SVM model to avoidthe over-fitting problem.

If the ratio is not above the threshold, steps 200-240 are repeated tofind a SVM model (with new kernel parameters and sometimes a new kernel)having a (T_best sample number)/(SVM support vector number) ratio thatexceeds the threshold (step 250). This iteration may involve trying andselecting different kernel parameters, and/or using a larger data set.

If such a SVM model having the new ratio above the threshold can befound in step 250, that SVM model is selected (step 260). Otherwise, ifsuch a ratio cannot be found, the computer processor selects the SVMmodel having kernel and the kernel parameters that has yielded thehighest ratio in step 250 (step 260).

The computer processor can then perform SVM predictions using theresulting SVM model on the entire data set of the feature matrix T_best(step 270). The resulting SVM can be the SVM found in steps 240 and 250that has the ratio exceeding the threshold, or the SVM model that hasyielded the highest ratio in step 260. In some embodiments, the featurevectors are based on faces or objects in sample images. The objects andthe faces in the sample images are more accurately classified by SVMpredictions using the resulting SVM model on T_best.

In some embodiments, FIG. 3 shows a second sub-flow of the disclosedmethods, in which the support vector set is further optimized in the SVMmodel obtained in the first sub-flow shown in FIG. 2. Specifically, thefirst sub-flow of the disclosed high-recall SVM method can be improvedfurther in accuracies by optimizing the placements of the featurevectors corresponding to the FP data set in vs. out of the featurematrix T_best. The accuracies of the SVM model is improved, while thecomplexity of the SVM model (number of support vectors) is reduced.

Referring to FIGS. 1-3, a feature vector v from the FP data set isselected and removed from the feature matrix T_best to produce a featurematrix T_v (step 300). The feature matrix T_v is a temporary featurematrix for evaluating the feature vector v.

The SVM model based on the kernel and the optimized kernel parametersadopted in step 260 is retrained on the feature matrix T_v to produce anew SVM model (step 310). The computer processor then performs SVMpredictions using the new SVM model on the feature matrix T_v (step320).

A benefit function value is then calculated (step 330) to evaluate theremoval of the feature vector v. The benefit function is dependent onthe differences between true positives, true negatives, and the numberof support vectors generated respectively by the latest SVM model onT_best and by the new SVM model on T_v (step 330).

For instance, an example of such benefit function is as follows:

C(v)=[size(FP_best)−size(FP_v)]*alpha+[size(FN_best)−size(FN_v)]*beta+[nSV_best−nSV_v]*gamma,

where alpha, beta and gamma are positive global parameters (i.e.weighting coefficients), used to control the ratio of sample number tosupport vector number. Decreases in False sets (Positive or Negative)and a decrease in model complexity (nSV, the number of support vectors)from T_best to T_v are beneficial, and will result in an increased valueof the benefit function C(v). In other words, a higher positive C(v)value indicates that it is beneficial to remove the feature vector v.

It should be noted that the benefic function C(v) can be equivalentlywritten using TP_best, TP_v, TN_best, and TN_v without affecting theoutcome of the present method:

C(v)=[−size(TP_best)+size(TP_v)]*alpha+[−size(TN_best)+size(TN_v)]*beta+[nSV_best−nSV_v]*gamma.

In other words, it is beneficial, as indicated by an increase of C(v),for an increase in True sets (Positive or Negative) and a decrease inmodel complexity (nSV, the number of support vectors) from T_best toT_v.

An example for the set of (alpha, beta, gamma) values are (1, 2, 5). Inthis example, alpha has an absolute value smaller than beta means thatFN is more significant than FP, and an increase in FN (i.e. a decreasein TN) should be penalized more, which is intended for the presenthigh-recall SVM method. A larger gamma represents a bias toward simpleSVM model with fewer support vectors and makes sure that support vectorsare removed if their removal simplifies the SVM model withoutsignificantly hampering performance.

The feature vector v is removed from T_best if the value of the benefitfunction meets a predetermined criterion (step 340). For example, thethreshold value can be zero for the benefit function. If C(v)>0, thenthe decrease in false sets is worthwhile and the model complexity hasnot jumped. If so, the new feature vector v is removed from T_best (step340). The True sets, False sets, and the feature vectors are updated inthe feature matrix:

-   -   FN_best=FN_v    -   TN_best=TN_v    -   FP_best=FP_v    -   TP_best=TP_v    -   nSV_best=nSV_v.        The updated T_best is the current best feature vector matrix        that has removed with the feature vectors v that has been        determined to be beneficial as describe above.

If the benefit function value does not meet the predetermined criterion,the feature vector v is not removed from the feature matrix T_best (step350). In the example above, if C(v)<=0, the True sets, False sets, andthe feature vectors in the feature matrix T_best are not updated.

Then steps 300-350 is repeated by selecting and removing a differentfeature vector corresponding to the FP data set from the feature matrixT_best (step 360). All the feature vectors corresponding to the FP dataset will be removed and evaluated in iteration by repeating steps300-360.

After all feature vectors corresponding to the FP data set have beenevaluated as discussed above, the computer processor performs SVMprediction using the feature matrix T_best that gives the highest bestfunction value (step 370). If no feature vector v has been a removedfrom T_best, the same version of T_best as in steps 260-270 is stillused for performing SVM predictions.

In some embodiments, the feature vectors are based on faces or objectsin sample images. The objects and the faces in the sample images aremore accurately classified by SVM predictions using the resulting SVMmodel on T_best.

After the faces and objects in the sample images are classified by theSVM model obtained in step 280 and step 370, the images can be properlyselected based on the classified faces and/or the objects and areincorporated in an image-product design. The classification of the facesand the objects, and the selections and incorporation of the photos canbe automated in the computer system to save users' time. The curationand layout of the image-product designs can be automatically created bythe computer processor 36, the computer device 60, or the mobile device61, then presented to a user 70 or 71 (FIG. 1), which allows the imageproduct to be ordered and manufactured by the printing and finishingfacilities 40 and 41 (FIG. 1). The image product creation can alsoinclude partial user input or selections on styles, themes, formats, orsizes of an image product, or text to be incorporated into an imageproduct.

The accurate recognition and grouping of faces and objects disclosedherein can significantly reduce time to create the product designs, andimprove the relevance and appeal of an image product. For example, themost important people can be determined and to be emphasized in an imageproduct. Redundant person's face images can be filtered out and selectedbefore incorporated into an image product. Irrelevant persons can beminimized or avoided in the image product.

The disclosed methods can include one or more of the followingadvantages. The disclosed methods can drastically increase theaccuracies in classifying objects and faces in images, especially insmall sample set. The disclosed methods can be implementedautomatically, and can produce image-based product designs with higherquality, better appeal, and more relevance to users.

It should be noted that the presently disclosed methods are not limitedto imaging applications and image-product designs. Examples of othersuitable applications include information classification and prediction,text recognition and classification, voice recognition, biometricsrecognition, classification of gene expression data and other biomedicaldata, etc.

What is claimed is:
 1. A computer system, comprising: one or morecomputer processors configured to select a kernel and kernel parametersfor a first Support Vector Machine (SVM) model that includes SVM supportvector number of support vectors, to test the first SVM model on afeature matrix T to produce false positive (FP) data set and falsenegative (FN) data set, wherein the feature matrix T includes n featurevectors of length m, wherein n and m are integer numbers, to produce aT_best using the feature matrix T, wherein the feature matrix T_bestcomprises T_best sample number of sample points; to check if a ratio(T_best sample number)/(SVM support vector number) is above a thresholdfor the first SVM model on the feature matrix T_best, and if the ratiois above the threshold, to perform SVM predictions using the first SVMmodel on the feature matrix T_best.
 2. The computer system of claim 1,wherein the n feature vectors are based on faces or objects in images,wherein the first SVM model is used on the feature matrix T_best toclassify the faces or the objects in the images.
 3. The computer systemof claim 2, wherein the one or more computer processors are furtherconfigured to create an image-product design based on the faces or theobjects in the images classified by the first SVM model using thefeature matrix T_best.
 4. The computer system of claim 1, wherein if theratio (T_best sample number)/(SVM support vector number) is not abovethe threshold for the first SVM model on the feature matrix T_best,repeating the steps of selecting, the one or more computer processorsare further configured to repeat the steps of testing, automaticallyremoving, retraining, and checking to find a second SVM model having(T_best sample number)/(SVM support vector number) ratio that exceedsthe threshold.
 5. The computer system of claim 4, wherein the one ormore computer processors are further configured to perform SVMpredictions using the second SVM model on the feature matrix T_best. 6.The computer system of claim 4, wherein if a SVM model having (T_bestsample number)/(SVM support vector number) ratio that exceeds thethreshold is not found, the one or more computer processors are furtherconfigured to select a third SVM model that yields a highest (T_bestsample number)/(SVM support vector number) ratio in the step ofrepeating the steps of selecting, testing, automatically removing,retraining, and checking; and to perform SVM predictions using the thirdSVM model on the feature matrix T_best.
 7. The computer system of claim1, wherein the threshold for the (T_best sample number)/(SVM supportvector number) ratio is ten or higher.
 8. The computer system of claim1, wherein the one or more computer processors are further configured toapply Grid Search on the feature matrix T to find optimal kernelparameters before the first SVM model on a feature matrix T is tested.9. The computer system of claim 1, wherein the feature vectors are basedon faces or objects in images, wherein the one or more computerprocessors are further configured to classify the objects and the facesin the images by performing SVM predictions on the feature matrixT_best.
 10. The computer system of claim 9, wherein the one or morecomputer processors are further configured to automatically createdesigns for photo products based on the objects and faces classified byperforming SVM predictions on the feature matrix T_best.
 11. Thecomputer system of claim 1, wherein the one or more computer processorsare further configured to select a feature vector v from the FP data setto remove from the feature matrix T_best to produce a feature matrixT_v, to retrain the first SVM model on the feature matrix T_v to producea fourth SVM model, to calculate a benefit function to produce a benefitfunction value, wherein the benefit function is dependent of differencesbetween false positives, false negatives, and numbers of support vectorsgenerated respectively on the feature matrix T_best by the first SVMmodel and on T_v by the fourth SVM model, to remove the feature vector vfrom the feature matrix T_best to obtain an updated feature matrixT_best if the benefit function value meets a predetermined criterion,and to perform SVM prediction using the fourth SVM model on the updatedfeature matrix T_best.
 12. The computer system of claim 11, wherein thebenefit function is dependent more strongly on a difference between thefalse negatives than on a difference between the false positives. 13.The computer system of claim 12, wherein the difference between falsenegatives has a first weighting coefficient in the benefit function,wherein the difference between false positives has a second weightingcoefficient in the benefit function, wherein the first weightingcoefficient is larger absolute value than the second weightingcoefficient.
 14. The computer system of claim 11, wherein the n featurevectors are based on faces or objects in images, wherein the fourth SVMmodel is used on the feature matrix T_best to classify the faces or theobjects in the images.
 15. The computer system of claim 14, wherein theone or more computer processors are further configured to create animage-product design based on the faces or the objects in the imagesclassified by the fourth SVM model using the feature matrix T_best. 16.The computer system of claim 11, wherein the one or more computerprocessors are further configured to repeat the steps of selecting afeature vector v, training the first SVM model, performing SVMpredictions, calculating a benefit function, and removing the featurevector v from the feature matrix T_best by selecting and removing adifferent feature vector corresponding to the FP data set from featurematrix T_best, wherein SVM predictions are performed using the featurematrix T_best that gives a highest best function value.
 17. The computersystem of claim 16, wherein all the feature vectors corresponding to theFP data set are evaluated by calculating a corresponding beneficfunction and determined to be removed from the feature matrix T_best ornot depending on a value of the corresponding benefic function.
 18. Thecomputer system of claim 11, wherein the feature vector v is not removedfrom the feature matrix T_best if the value of the benefit function doesnot meet the predetermined criterion.
 19. The computer system of claim11, wherein the feature vectors are based on faces or objects in images,wherein the one or more computer processors are further configured toclassify the objects and the faces in the images by performing SVMpredictions on the feature matrix T_best.